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ABSTRACT 



A method, system and computer program product for updat- 
ing firmware in a disk in a disk array in a RAID storage 
system without deactivating a server coupled to the disk 
array. Upon selecting the disk whose firmware is to be 
updated, the selected disk may enter a degrade mode of 
operation where a RAID controller coupled to the RAID 
storage system may prevent requests from reaching the 
selected disk thereby suppressing activity on the selected 
disk to allow the firmware to be updated. During the 
updating of the firmware, any stripes updated may be 
tracked. Upon completion of the firmware update, the stripe 
units in the selected disk associated with the stripes updated 
may be rebuilt. In this manner, firmware may be updated on 
a disk in a RAID storage system without deactivating the 
server thereby allowing the server to continuously stay 
active. 

81 Claims, 12 Drawing Sheets 
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FIG. 5 
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FIG. 6 
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FIG. 6 (cont'd) 
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FIG. 8 
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UPGRADING FIRMWARE ON DISKS OF 
THE RAID STORAGE SYSTEM WITHOUT 
DEACTIVATING THE SERVER 

TECHNICAL FIELD 

The present invention relates to the field of a Redundant 
Array of Independent Disks (RAID) storage system, and 
more particularly to upgrading firmware on the disks in the 
RAID without deactivating the server coupled to the RAID 
storage system. 

BACKGROUND INFORMATION 

As the performance of microprocessor and semiconductor 
memory technology improves, there is a need for improved 
data storage systems with comparable performance 
enhancements. Additionally, in enhancing the performance 
of data storage systems, there is a need for improved 
reliability of data stored. In 1988, a paper was published by 
Patterson, Gibson, Katz, A Case for Redundant Arrays of 
Independent Disks (RAID), International Conference on 
Management of Data, pgs. 109-116, June 1988. This paper 
laid the foundation for the use of redundant arrays of 
independent disks that would not only improve the data 
transfer rate and data I/O rate over a comparable single disk 
access, but would also provide error correction at a lower 
cost in data storage systems. 

RAID may include an array of disks which may be 
coupled to a network server. The server, e.g., file server, 
database server, web server, may be configured to receive a 
stream of requests (Input/Output (I/O) requests) from clients 
in a network system to read from or write to particular disks 
in the RAID, The I/O requests may also be issued from an 
application within the server. The server may comprise a 
RAID controller which may be a hardware and/or software 
tool for providing an interface between the server and the 
array of disks. The server may forward the I/O requests to 
the RAID controller which may retrieve or store the 
requested data. Topically, the RAID controller manages the 
array of disks for storage and retrieval and views the disks 
of the RAID separately. The disks included in the array may 
be any type of data storage systems which may be controlled 
by the RAID controller when grouped in the array. 

The RAID controller may typically be configured to 
access the array of disks as defined by a particular "RAID 
level." The RAID level may specify how the data is distrib- 
uted across the disk drives and how error correction is 
accomplished. In the paper noted above, the authors describe 
five RAID levels (RAID Level 1-RAID Level 5). Since the 
publication of the paper, additional RAID levels have been 
designated. 

RAID levels are typically distinguished by the benefits 
included. Three key benefits which may be included in a 
RAID level are fault tolerance, data availability and high 
performance. Fault tolerance may typically be achieved 
through an error correction method which ensures that 
information can be reconstructed in the event of a disk 
failure. Data availability may allow the data array to con- 
tinue to operate with a failed component. Typically, data 
availability may be achieved through a method of redun- 
dancy. Finally, high performance may typically be achieved 
by simultaneous access to multiple disk drives which results 
in faster I/O and data transfer requests. 

Error correction may be accomplished, in many RAID 
levels, by utilizing additional parity data stored with the 
original data. Parity data may be utilized to recover lost data 
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due to disk failure. Parity data may typically be stored on 
one or more disks dedicated for error correction only or 
distributed over all of the disks within an array. 

By the method of redundancy, data may be stored in 

5 multiple disks of the array. Redundancy is a benefit in that 
redundant data allows the storage system to continue to 
operate with a failed component while data is being replaced 
through the error correction method. Additionally, redundant 
data is more beneficial than backup data because back-up 

10 data is typically outdated when needed whereas redundant 
data is current when needed. 

In many RAID levels, redundancy may be incorporated 
through data interleaving which distributes the data over all 
of the data disks in the array. Data interleaving is usually in 

15 the form of data "striping" in which data to be stored is 
broken down into blocks called "stripe units" which are then 
distributed across the array of disks. Stripe units are typi- 
cally predefined as a bit, byte, block or other unit. Stripe 
units are further broken into a plurality of sectors where all 

20 sectors are an equivalent predefined size. A "stripe" is a 
group of corresponding stripe units, one stripe unit from 
each disk in the array. Thus, "stripe size" is equal to the size 
of a stripe unit times the number of data disks in the array. 
In an example, RAID level 5 utilizes data interleaving by 

25 striping data across all disks and provides for error correc- 
tion by distributing parity data across all disks. For each 
stripe, all stripe units are logically combined with each of the 
other stripe units to calculate parity for the stripe. Logical 
combination may be accomplished by an exclusive or 

30 (XOR) of the stripe units. For N physical drives, N-l of the 
physical drives will receive a stripe unit for the stripe and the 
Nth physical drive will receive the parity for the stripe. For 
each stripe, the physical drive receiving the parity data 
rotates such that all parity data is not contained on a single 

35 disk. 

Disk arrays may be configured to include logical drives 
which divide the physical drives in the disk array into logical 
components. Each logical drive may include a cross section 
of each of the physical drives and may be assigned a RAID 
40 level. 

Each disk in the disk array of a RAID may store firmware 
where firmware may refer to software that may be burned 
into a memory chip, e.g., Read-Only Memory (ROM), 
Programmable ROM (PROM), Erasable PROM (EPROM), 

45 Electrical EPROM (EEPROM) or into the hard drive. 
Typically, the firmware stored on the disk in the disk array 
may be configured to perform functions such as sector 
re-mapping, monitoring the disk for failures, etc. 
In order to update the firmware stored on a disk in the disk 

50 array, all activity on that disk must become inactive. If the 
disk receives a request to read from or write to that disk 
during the process of upgrading the firmware, the disk may 
become inoperative either permanently or temporarily. 
Subsequently, the server coupled to the RAID comprising 

55 the disk with the firmware to be updated must be deactivated 
thereby assuring that the disk does not receive any I/O 
requests. 

However, some servers such as mission critical servers 
may be required to stay active continuously. It would 
60 therefore be desirable to be able to update firmware on the 
disks in a RAID storage system without deactivating the 
server coupled to the RAID thereby allowing the server to 
continuously stay active. 

6S SUMMARY 

The problems outlined above may at least in part be 
solved in some embodiments by selecting a disk in a disk 
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array of the RAID storage system to have its firmware A determination may then be made as to whether the 

updated. The selected disk may enter a degrade mode of updating of the firmware on the selected disk is completed, 

operation where the RAID controller coupled to the RAID If the updating of the firmware is not completed, then a 

may prevent requests from reaching the selected disk determination may be made as to whether the RAID con- 

thereby suppressing activity on the selected disk to allow the 5 troller received any read requests for the stripe unit in the 

firmware to be updated. During the updating of the selected disk. Upon completion of the firmware being 

firmware, any stripes updated may be tracked. Upon updated on the selected disk, the stripe units in the selected 

completion of the firmware update, the stripe units in the disk associated with any stripes to be updated may be rebuilt 

selected disk associated with stripes updated may be rebuilt. as described further below. 

In this manner, firmware may be updated on a disk in a 10 if the updating of the firmware is complete, then a 

RAID storage system without deactivating the server determination may be made as to whether any stripes were 

coupled to the RAID storage system thereby allowing the updated. If there were no stripes updated, then there is no 

server to continuously stay active. need to Kbu ^ any stripe in the selected disk. 

In one embodiment of the present invention, a method for If there were stripes up d a ted, then a stripe unit associated 

updating firmware on a disk in a RAID storage system 15 with a stripe updated may be rebuilt. In one embodiment, the 

implementing a RAID level one system without deactivating stripe units i n the selected disk associated with stripes that 

a server coupled to the RAID storage system may comprise have been updated may be rebuilt stripe by stripe updated 

the step of selecting a particular physical disk in the RAID starting from the top stripe that was updated to the bottom 

storage system to update the firmware in that particular disk. s t ripe> if anV) that was updated. In one embodiment, the 

The selected disk as well as the associated logical drives 20 str i pe unit associated with a stripe updated may be rebuilt by 

may enter a degrade mode of operation. In the degrade mode copying data in the stripe unit that mirrors the stripe unit to 

of operation, a RAID controller, providing an interface be rebuilt and storing that data in the stripe unit to be rebuilt. 

between the server and the RAID storage system, may 0 _ . - , , ... tU , . . t . , 

, ... & ' - . ' Each stnpe associated with the stnpe unit rebuilt may be 

suppress particular activities, e.g., recovery actions, hot , , , A f t . 4 - tU . r . . i_.i_ .i_ 

, . . • r ' J? i . jj-i ii nc tracked. A determination may then be made as to whether the 

spare kickin, from occurring on the selected disk as well as 25 n ATn . , ' . 

. . ai ■< .-jx*_ RAID controller received a write request for a stnpe unit 

prevent requests, e.g., read/ wnte requests rebuilt in the selected disk prior to completing the rebuilding 

cuent, hot swap queries, from reaching the selected disk. By f M ^ ^ ^ be ^J, 

suppressing particular activities from occurring on the r n 

selected disk and preventing requests from reaching the If the RAID controller did not receive a wnte request for 

selected disk, activity on the selected disk may become 30 a stnpe unit rebuilt in the selected disk pnor to completing 

inactive thereby allowing the firmware on the selected disk ±c rebuilding of all stripe units required to be rebuilt, then 

to be updated a determination may be made as to whether another disk in 

The firmware on the selected disk may then be updated. ?» dis *. . of Jj» RAID s ' ora f .f** 1 ™ bas . becomc 

During the updating of the firmware, the following may ! D °P erat | ve dunng the process of rebuilding the stripe units 

occur 35 in the selected disk required to be rebuilt as discussed further 

A determination may be made as to whether the RAID TC ' nirT ^ „ . „ . 

controller received any read requests for the stripe units in ' ! f lh * controller received a wnte request for a stripe 

the selected disk. If the RAID controller received a request u f ,^ bmlt . select ? d disk P™ }° coveting the 

to read data stored in a stripe unit in the selected disk, then »builduig of all stnpe units required to be rebuilt, then the 

the RAID controller may retrieve and transmit the requested 40 *fmiy be wntten in both the stripe unit rebuilt in 

data stored in the stripe unit that mirrors the stripe unit me sel ? ct ^ d disk and in stn P e umt that mirrors me stn P e 

containing the requested data. umt rebuUt * 

If the RAID controller did not receive a request to read . A determination may then be made as to whether another 

data stored in the stripe unit in the selected disk, then a A , dlsk m * e dlsk arra y of the RAID stora S e svsteirj has 

determination may be made as to whether the RAID con- become J DO P crat i v e during the process of rebuilding the 

troller received any write requests for the stripe units in the stn P e umts m the selected dlsk re£ - uired t0 be rebuilt 

selected disk. Furthermore, upon transmitting the requested If another disk in the disk array of the RAID storage 

data stored in the stripe unit that mirrors the stripe unit system does not become inoperative during the process of 

containing the requested data, a determination may be made rebuilding the stripe units in the selected disk required to be 

as to whether the RAID controller received any write rebuilt, then a determination may be made as to whether the 

requests for the stripe units in the selected disk. rebuilding of the stripe units whose data has changed has 

If the RAID controller did not receive any write requests been completed as discussed further below, 

for the stripe units in the selected disk, then a determination If another disk in the disk array of the RAID storage 

may be made as to whether the updating of the firmware is 55 system has become inoperative during the process of 

complete as discussed further below. rebuilding the stripe units in the selected disk required to be 

If the RAID controller received any write requests for the rebuilt, then the stripe units in the selected disk required to 

stripe units in the selected disk, then the RAID controller be rebuilt ma y be rebuilt usin S a C0 Py of updated data 

may write the updated data in the stripe unit that mirrors the stored for backup purposes. 

stripe unit containing the data that was updated. A copy of 60 A determination may be made as to whether the rebuild- 

the updated data may be stored for backup purposes. ing of the stripe units whose data has changed has been 

The stripe associated with the stripe units in the selected completed. If not, then the next stripe unit required to be 

disk whose data was changed may be tracked. In one rebuilt may be rebuilt. If the rebuilding of the stripe units in 

embodiment, the stripes associated with the stripe units in the selected disk whose data has changed has been 

the selected disk whose data was changed may be tracked in 65 completed, then the method may be terminated, 

a table stored in a non-volatile memory of the RAID It is noted that even though the above method describes a 

controller. method for updating firmware on a disk in the RAID storage 
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system implementing a RAID level one system without the selected disk whose data was changed may be tracked in 

deactivating the RAID controller that the principles a table stored in a non-volatile memory of the RAID 

described in the above method may be applicable to any controller. 

redundant RAID level. It is further noted that a person of A determination may be made if the updating of the 

ordinary skill would be capable of applying the principles 5 firmware on the selected disk is completed. If the updating 

taught in the above method to any redundant RAID level. of the firmware is not completed, then a determination may 

In another embodiment of the present invention, a method be made as t0 whether the RAID controller received any 

for updating firmware on a disk in a RAID storage system read «Wts to th fl e stri P e uoits * ^ ^fcted disk - , U P°° 

implementing a RAID level five system without deactivating ™m p IT? • lhe t firmWare "? 6 ** d °? * e 

a RAID controller coupled to the RAID storage system may 10 ^ th * uml f m the d f ™^ * ny 

lL 4 r i ^-i l ■ i j- i • stripes to be updated may be rebuilt as described further 

comprise the step of selecting a particular physical disk in below 

the RAID storage system implementing a RAID level five T r , c 4 . ^ . , , t , 

j * *u c ■ iL 4 4 . , j • i If the updating of the firmware is complete, then a 

system to update the firmware in that particular disk. determination may be made as to whether any stripes were 

The selected disk as well as the associated logical drives updated. If there were no stripes updated, then there is no 

may enter a degrade mode of operation. In the degrade mode 15 nee d to rebuild any stripe units in the selected disk, 

of operation, the RAID controller may suppress particular if there were stripes updated, then a stripe unit associated 

activities, e.g., recovery actions, hot spare kickin, from with a stripe updated may be rebuilt. In one embodiment, the 

occurring on the selected disk as well as prevent requests, stripe units in the selected disk associated with stripes that 

e.g., read/write requests issued from a client, hot swap have been updated may be rebuilt stripe by stripe updated 

queries, from reaching the selected disk. By suppressing 20 starting from the top stripe that was updated to the bottom 

particular activities from occurring on the selected disk and stripe, if any, that was updated. In one embodiment, the 

preventing requests from reaching the selected disk, activity stripe unit associated with a stripe updated may be rebuilt by 

on the selected disk may become inactive thereby allowing performing a logical calculation on data located in other 

the firmware on the selected disk to be updated. stripe units of the stripe associated with the stripe unit 

The firmware on the selected disk may be updated. During 25 containing the outdated data. The resulting data may then be 

the updating of the firmware on the selected disk, the mserted m the stn P e umt t0 be rebuUt ' 

following may occur ^ determination may then be made as to whether another 

A determination may be made as to whether the RAID j£ S^dL^ 'foe^^ of TebuflcT "h* 

controller received any read requests for the stripe units in ? *u l i ^^^i- i 6 P roces ? 0 , re u , . 1D & e 

4 , i jc ii_ a ¥t\ a it j 30 stripe units in the selected disk required to be rebuilt. 

the selected disk. If the RAID controller received a request S t , , . - . n c , n , m 

_ . . i * j j * i *u " another disk in the disk array of the RAID storage 

to read data stored in a stripe unit m the selected disk, then 4 . „ . j ■ *i. r 

(U nAm . „ v _c i-iii. system does not become inoperative during the process of 

the RAID controller may perform a logical calculation on „u„:m: # l„ , ■ H i * a a- i • a * u 

j . i 4 , . . . v .■.•it . rebuilding the stnpe units m the selected disk required to be 

data located in other stnpe units associated with the stripe rcbuilt> theD a determination may be made as to whether the 

unit containing the data requested. The resulting data may 35 rebuilding of the stripe units whose data has changed has 

then be transmitted to the requesting client. been completed as discussed further below. 

If the RAID controller did not receive a request to read if another disk in the disk array of the RAID storage 

data stored in the stripe unit in the selected disk, then a system has become inoperative during the process of 

determination may be made as to whether the RAID con- rebuilding the stripe units in the selected disk required to be 

troller received any write requests for the stripe units in the 4Q rebuilt, then the stripe units in the selected disk required to 

selected disk. Furthermore, upon transmitting the requested be rebuilt may be rebuilt using a copy of the updated data 

data, a determination may be made as to whether the RAID stored for backup purposes. 

controller received any write requests for the stripe units in A determination may then be made as to whether the 

the selected disk. rebuilding of the stripe units whose data has changed has 

If the RAID controller did not receive any write requests 45 been completed. If not, then the next stripe unit required to 

for the stripe units in the selected disk, then a determination be rebuilt may be rebuilt. If the rebuilding of the stripe units 

may be made as to whether the updating of the firmware is in the selec ted disk whose data has changed has been 

complete as discussed further below. completed, then the method may be terminated. 

If the RAID controller received any write requests for the " is noted that even though the above method describes a 

stripe units in the selected disk, then the RAID controller 50 f'^l^^^ 

maygenerateanewparity^ T^T^^l^ ^T™*^ 

s ripe unit contammg outdated data A new parity for he describcd £ ^ ^ mcthod ^ u P P 

stripe associated with the stape unit containing outdated redundant ^ ^ ft fa ^/^Xt a person of 

data may be generated so that the updated data may replace Qrdm ^ would fae Wc f £ 

the outdated data as explained further below. 55 Uught ? ^ ^ mc{h J |Q ^ ^ ^ ^ 

In one embodiment, a new parity may be generated by ^ f { hasoullmed rather broadly the features and 

performing a logical operation on the data to be written Stages of one 0f more of the 

along with the data stored m the other stripe units except the * Qrder ^ ^ 

stripe unit storing ^ Parity data to be updated and the stripe ^ follows be ^rstood Additional 

unit of the selected disk whose firmware is being updated. 60 features md advanta es * f the ^ be described 

? h P ,h . H P3n ^ "^h f n H r T , t^VJT b «einafter which form the subject of the claims of the 

ciated with the stnpe updated. A copy of the updated data, invention 
i.e., the data requested to be written in the stripe units in the 

selected disk, may be stored for backup purposes. BRIEF DESCRIPTION OF THE DRAWINGS 

The stripe associated with the stripe unit in the selected 65 A better understanding of the present invention can be 

disk whose data was changed may be tracked. In one obtained when the following detailed description is consid- 

embodiment, the stripes associated with the stripe units in ered in conjunction with the following drawings, in which: 
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FIG. 1 illustrates a network system configured in accor- ment for practicing the present invention. Client 110 may 

dance with the present invention; have a central processing unit (CPU) 210 coupled to various 

FIG. 2 illustrates an embodiment of the present invention other components by system bus 212. An operating system 

of a client in the network system; 240, may run on CPU 210 and provide control and coordi- 

FIG. 3 illustrates an embodiment of the present invention 5 nate the Actions of the various components of FIG. 2. An 

of a server in the network system; application 250 m accordance with the principles of the 

FIG. 4 illustrates a RAID controller configured in accor- P^sent invention may run in conjunction with operating 

dance with the present invention; system 240 and provide calls to operating system 240 where 

m .,, , , _ the calls implement the various functions or services to be 

FIG. 5 illustrates an exemplary disk array of a RAID JQ formed b app i icatioQ 25 0. Read only memory (ROM) 

storage system implementing RAID level one; n6 may be coupled tQ system bus in ^ a ^ 

FIG. 6 is a flowchart of a method for updating firmware input/output system ("BIOS") that controls certain basic 

on a disk in the RAID storage system implementing RAID functions of client 110. Random access memory (RAM) 

level one; 214, I/O adapter 218 and communications adapter 234 may 

FIG. 7 illustrates an exemplary disk array of a RAID is also be coupled to system bus 212, It should be noted that 

storage system implementing RAID level five; and software components including operating system 240 and 

FIG. 8 is a flowchart of a method for updating firmware application 250 may be loaded into RAM 214 which may be 

on a disk in the RAID storage system implementing RAID the computer system's main memory. I/O adapter 218 may 

level five. be a small computer system interface ("SCSI") adapter that 

20 communicates with a disk unit 220, e.g., disk drive. 

DETAILED DESCRIPTION Communications adapter 234 may interconnect bus 212 

It is noted that even though the following describes the with an outside network enabling client 110 to communicate 

present invention implementing either a RAID level one or with server 120 via a Local Area Network (LAN), e.g., 

a RAID level five implementation, the present invention Ethernet, Token Ring, ARCnet or a Wide Area Network 

may be implemented using any redundant RAID level. It is 25 (WAN), e.g., Internet. Input/Output devices may also be 

further noted that a person of ordinary skill in the art would connected to system bus 212 via a user interface adapter 222 

be capable of applying the principles taught in the figures and a display adapter 236. Keyboard 224, mouse 226 and 

and description describing the updating of the firmware on speaker 230 may all be interconnected to bus 212 through 

a disk in the RAID storage system without deactivating the user interface adapter 222. Event data may be inputted to 

server coupled to the RAID storage system implementing 30 client 110 through any of these devices. A display monitor 

either a RAID level one or a RAID level five to any 238 may be connected to system bus 212 by display adapter 

redundant RAID level. 236. In this manner, a user is capable of inputting to client 

FIG. 1 — Network System 110 through keyboard 224 or mouse 226 and receiving 

FIG. 1 illustrates one embodiment of the present inven- output from client U0 via display 238 or speaker 230. 

tion of a network system 100. Network system 100 may 35 FIG. 3 — Server 

comprise one or more clients 110A-D coupled to a server FIG. 3 illustrates a typical hardware configuration of 

120, e.g., file server, database server, web server. Server 120 server 120 which is representative of a hardware environ- 

may be coupled to a Redundant Array of Independent Disks ment for practicing the present invention. Server 120 may 

(RAID) storage system 130. RAID storage system 130 may employ a peripheral component interconnect (PCI) local bus 

comprise an array of physical disks as explained in greater 40 architecture. Although the depicted example employs a PCI 

detail in conjunction with FIGS. 5 and 7. Clients 110A-D bus, other bus architectures such as Accelerated Graphics 

may collectively or individually be referred to as clients 110 Port (AGP) and Industry Standard Architecture (ISA) may 

or client 110, respectively. Each client 110 may be config- be used. Processor 301 and main memory 302, e.g., 

ured to issue requests, e.g., read/write requests, to server 120 Dynamic Random Access Memory (DRAM), Synchronous 

to read from or write to a particular disk in RAID storage 45 DRAM (SDRAM), may be connected to PCI local bus 303 

system 130. A more detailed description of client 110 is through north bridge 304. North bridge 304 may also include 

provided below in conjunction with FIG. 2. Server 120 may an integrated memory controller and cache memory for 

be configured to forward the requests to a RAID controller processor 301. Furthermore, an operating system 305 may 

to either retrieve from the requested disk in RAID storage run on processor 301 to provide control and coordinate the 

system 130 the requested information or store in the 50 functions of the various components of FIG. 3. An applica- 

requested disk in RAID storage system 130 the requested tion 306 in accordance with the principles of the present 

information. A more detailed description of server 120 is invention may run in conjunction with operating system 305 

provided further below in conjunction with FIG. 3. It is and provide calls to operating system 305 where the calls 

noted that system 100 may comprise any number of clients implement the various functions or services to be performed 

110 and servers 120 and that FIG. 1 is illustrative. It is 55 by operating system 305 for application 306. It should be 

further noted that in another embodiment, system 100 may noted that software components including operating system 

only comprise server 120 coupled to RAID storage system 305 and application 306 may be loaded into system's main 

130 where an application in server 120 instead of client 110 memory 302. 

generates requests, e.g., read/write requests, to read from or RAID controller 307 and network interface connection 

write to a particular disk in RAID storage system 130. It is 60 308 may also be connected to PCI local bus 303. RAID 

further noted that network system 100 may be any type of controller 307 may be coupled to RAID storage system 130 

system such as a file system or a database system and that thereby providing an interface between server 120 and 

FIG. 1 is not to be limited in scope to any one particular RAID storage system 130. RAID controller 307 may be 

embodiment. configured to receive requests, e.g., read/write requests, to 

FIG. 2 — Client 65 read from or write to a particular disk in RAID storage 

FIG. 2 illustrates a typical hardware configuration of system 130. A more detailed description of RAID controller 

client 110 which is representative of a hardware environ- 307 is provided below in FIG. 4. Network Interface con- 
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iroller 308 may interconnect PCI bus 303 with an outside 
network enabling server 120 to communicate with clients 
110 or RAID storage system 130 via a Local Area Network 
(LAN), e.g., Ethernet, Token Ring, ARCnet or a Wide Area 
Network (WAN), e.g., Internet. 5 

Those of ordinary skill in the art will appreciate that the 
hardware in FIG. 3 may vary depending on the implemen- 
tation. Other internal hardware or peripheral devices, such as 
flash ROM (or equivalent non -volatile memory) or optical 
disk drives and the like, may be used in addition to or in 10 
place of the hardware depicted in FIG. 3. It is noted that the 
depicted example in FIG. 3 and the above described example 
are not meant to imply architectural limitations. 

Implementations of the invention include implementa- 
tions as a computer system programmed to execute the is 
method or methods described herein, and as a computer 
program product. According to the computer system 
implementations, sets of instructions for executing the 
method or methods may be resident in main memory 302 of 
one or more computer systems configured generally as 20 
described above. Until required by server 120, the set of 
instructions may be stored as a computer program product in 
another computer memory. Furthermore, the computer pro- 
gram product may be stored at another computer and trans- 
mitted when desired to the user's workstation by a network 25 
or by an external network such as the Internet. One skilled 
in the art would appreciate that the physical storage of the 
sets of instructions physically changes the medium upon 
which it is stored so that the medium carries computer 
readable information. The change may be electrical, 30 
magnetic, chemical or some other physical change. 
FIG. 4— RAID Controller 

FIG. 4 illustrates an embodiment of RAID controller 307 
configured in accordance with the present invention. As 
depicted, RAID controller 307 may comprise a processor 35 
401 configured to control RAID storage system 130. Pro- 
cessor 401 may preferably be an advanced microprocessor 
which may be coupled to processor bus 402. As depicted, 
also coupled to processor bus 402 is code/data RAM 403 
which may be utilized to temporarily store code and data 40 
utilized by processor 401. ROM 404 and non-volatile 
memory 405 may be accessed utilizing an 8-bit bus and thus 
bus interface 406 may be utilized to interface those devices 
to processor bus 402, which typically utilizes a 32-bit bus. 

Operational code may be stored within ROM 404, which, 45 
as those skilled in the art will appreciate, is generally 
provided utilizing so-called "flash" ROM. Operational code 
may thereafter be fetched from ROM 404 by processor 401 
upon initiation of operation to direct the operation of pro- 
cessor 401 to perform functions including the functions of 50 
the present invention. Non-volatile memory 405 may be a 
low power CMOS memory which is powered up for "back- 
up" by a battery such that the information stored in non- 
volatile memory 405 will not be lost when main power is 
terminated. Thus, non-volatile memory 405 may be utilized 55 
to store configuration data or operational code in a manner 
similar to that stored within ROM 404. ROM 404 may 
generally be updated at initial power application and any 
changes to system configuration during operation may be 
stored within non-volatile memory 405 and then entered into 60 
a "device change list" which may also be stored within 
non-volatile memory 405 and on each disk drive within the 
system. Furthermore, non-volatile memory 405 may store 
the program for updating firmware in a disk in RAID storage 
system 130 implementing either a RAID level one or a 65 
RAID level five without deactivating server 120 as 
described in conjunction with the description of FIGS. 5-8. 



It is noted that the program of the present invention that 
updates firmware in a disk in RAID storage system 130 
implementing either a RAID level one or a RAID level five 
without deactivating server 120 as described in conjunction 
with the description of FIGS. 5-8 may reside in other 
memory units, e.g., ROM 404, or a driver associated with 
operating system 305 of server 120. Furthermore, non- 
volatile memory 405 may be configured to store a table used 
for tracking any stripes that have been updated in the 
physical disk of RAID storage system 130 whose firmware 
is being updated as described in greater detail in conjunction 
with FIGS. 5-8. 

A non-volatile cache 407 may be coupled to DRAM/XOR 
controller 408. Non-volatile cache 407 may be configured 
into a plurality of temporary storage positions for data. That 
is, non-volatile cache 407 may be configured to store a copy 
of the updated data to be stored in the physical disk of RAID 
storage system 130 whose firmware is being updated as 
described in greater detail in conjunction with FIGS. 7-8. 
DRAM/XOR controller 408 may be utilized to control 
access to random access memory and also provides a 
hardware implemented exclusive or (XOR) circuit which 
may be utilized to rapidly and efficiently calculate parity for 
changes in updated data as well as utilized to perform the 
calculations discussed in FIG. 8. 

DRAM/XOR controller 408 may be coupled to local bus 
409. Also coupled to local bus 409 may be one or more small 
computer system interface (SCSI) control chips 410. Each 
SCSI control chip 410 may be defined as including channels 
which may each support RAID storage system 130. 

RAID controller 307 may further comprise a PCI bus 
adapter 411 configured to interconnect RAID controller 307 
with PCI bus 303 (FIG. 3). 

Those having ordinary skill in this art will appreciate that 
FIG. 4 is illustrative and that FIG. 4 is not meant to imply 
architectural limitations. 
FIG. 5 — RAID Level One 

FIG. 5 illustrates RAID storage system 130 implementing 
RAID level one which uses mirroring in order to store 
identical data on two disks. Referring to FIG. 5, the disk 
array in RAID storage system 130 implementing RAID level 
one may comprise physical disks 501A-B. Physical disks 
501 A-B may collectively or individually be referred to as 
physical disk 501. Each physical disk 501 may comprise a 
plurality of units commonly referred to as stripe units 
502A-J. For example, physical disk 501A may comprise 
stripe units 502A-E. Physical disk 501B may comprise 
stripe units 502F-J. Stripe units 502A-J may collectively or 
individually be referred to as stripe units 502 or stripe unit 
502, respectively. Each stripe unit 502 may be configured to 
store a predefined amount, e.g., bit(s), byte(s), of data. It is 
noted that each physical disk 501 may comprise any number 
of stripe units 502. It is further noted that RAID level one 
may comprise any number of physical disks 501 and that 
FIG. 5 is illustrative. 

A stripe 503A-E may refer to a group of corresponding 
stripe units 502. For example, stripe 503 A may include 
stripe units A and A'. Stripe 503B may include stripe units 
B and B' and so forth. Stripes 503A-E may collectively or 
individually be referred to as stripes 503 or stripe 503. 

The disk array in RAID storage system 130 implementing 
RAID level one may include logical drives which divide 
physical drives 501 in the disk array into logical components 
which may be viewed by client 110 as separate drives. Each 
logical drive may include a cross section of each of physical 
drives 501. For example, logical drive 504 may include 
stripes 503A-E across physical drives 501A-B. 
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FIG. 6 — Method for Updating Firmware on a Disk In a a determination may be made as to whether RAID controller 

RAID Level One System 307 received any write requests for stripe units 502 in 

FIG. 6 is a flowchart of one embodiment of the present selected disk 501, e.g., physical disk 501A. Furthermore, 

invention of a method 600 for updating firmware on a disk upon transmitting the requested data stored in stripe unit 

in RAID storage system 130 implementing a RAID level 5 502, e.g., stripe unit 502F, that mirrors stripe unit 502, e.g., 

one system without deactivating server 120. As stated in the stripe unit 502 A, containing the requested data in step 605, 

Background Information section, each disk in the disk array a determination may be made in step 606 as to whether 

of a RAID may store firmware where firmware may refer to RAID controller 307 received any write requests for stripe 

software that may be burned into a memory chip, e.g., Read units 502 in the selected disk 501, e.g., physical disk 501 A. 

Only Memory (ROM), Programmable ROM (PROM), Eras- 10 If RAID controller 307 did not receive any write requests 

able PROM (EPROM), Electrical EPROM (EEPROM), or for stripe units 502 in selected disk 501, e.g., physical disk 

into the hard drive. In order to update the firmware stored on 501 A, then a determination may be made in step 610 as to 

a disk in the disk array of a RAID, all activity on that disk whether the updating of the firmware is complete as dis- 

must become inactive. If the disk receives a request to read cussed further below. 

from or write to that disk during the process of upgrading the 15 If RAID controller 307 received any write requests for 

firmware, the disk may become inoperative either perma- stripe units 502 in the selected disk 501, e.g., physical disk 

nently or temporarily. Subsequently, the server coupled to 501 A, then, in step 607, RAID controller 307 may write the 

the RAID comprising the disk with the firmware to be updated data in stripe unit 502, e.g., stripe unit 502F, that 

updated must be deactivated thereby assuring that the disk minors stripe unit 502, e.g., stripe unit 502A, containing the 

does not receive any I/O requests. However, some servers 20 data that was updated. For example, if client 110, e.g., client 

such as mission critical servers may be required to stay 110A, requested to write over the data labeled "A" in stripe 

active continuously. It would therefore be desirable to be unit 502A during updating of the firmware on physical disk 

able to update firmware on the disks in a RAID storage 501 A, then RAID controller 307 may write over the data 

system without deactivating the server coupled to the RAID labeled "A" in stripe unit 502F that mirrors stripe unit 502 A 

thereby allowing the server to continuously stay active. 25 to store the updated data. If client 110, e.g., client 11 OA, 

Method 600 is a method for updating firmware on a disk in requested to write over the data labeled "D" in stripe unit 

RAID storage system 130 implementing a RAID level one 502A during updating of the firmware on physical disk 

system without deactivating server 120. 501 A, then RAID controller 307 may write over the data 

Referring to FIGS. 5 and 6, in step 601, a particular labeled "D'" in stripe unit 502D that mirrors stripe unit 502 A 

physical disk 501, e.g., physical disk 501A, in RAID storage 30 to store the updated data. 

system 130 (FIGS. 1 and 3) implementing a RAID level one In step 608, a copy of the updated data may be stored for 

system may be selected to update the firmware in that backup purposes. In one embodiment, a copy of the updated 

particular disk 501. data may be stored in a non-volatile cache 407 (FIG. 4) in 

In step 602, the selected disk 501, e.g., physical disk RAID controller 307. 

501A, as well as the associated logical drives 504 (FIG. 5) 35 In step 609, stripe 503, e.g., stripe 503 A, stripe 503D, 

enter a degrade mode of operation. In the degrade mode of associated with stripe units 502, e.g., stripe unit 502 A, stripe 

operation, RAID controller 307 (FIGS. 3 and 4) may sup- unit 502D, in the selected disk 501, e.g., physical disk 501 A, 

press particular activities, e.g., recovery actions, hot spare whose data was changed may be tracked. In one 

kickin, from occurring on selected disk 501 as well as embodiment, stripes 503 associated with stripe units 502 in 

prevent requests, e.g., read/write requests issued from client 40 the selected disk 501 whose data was changed may be 

110 (FIG. 1), hot swap queries, from reaching selected disk tracked in a table stored in non-volatile memory 405 (FIG. 

501. By suppressing particular activities from occurring on 4) of RAID controller 307. 

the selected disk 501 and preventing requests from reaching In step 610, a determination may be made if the updating 

selected disk 501, activity on the selected disk 501 may of the firmware on the selected disk 501, e.g., physical disk 

become inactive thereby allowing the firmware on the 45 501 A, is completed. Upon completion of the firmware being 

selected disk 501 to be updated. updated on the selected disk 501, e.g., physical disk 501A, 

In step 603, the firmware on the selected disk 501, e.g., the stripe units 502 in the selected disk 501, e.g., physical 

physical disk 501 A, may be updated. disk 501A, associated with any stripes to be updated, may be 

During the updating of the firmware on the selected disk rebuilt as described further below in steps 613-619. 

501, e.g., physical disk 501A, the following steps 604-610 so If the updating of the firmware is not completed, then a 

may occur. In step 604, a determination may be made as to determination may be made as to whether RAID controller 

whether RAID controller 307 received any read requests for 307 received any read requests for stripe units 502 in the 

stripe units 502 in the selected disk 501, e.g., physical disk selected disk 501, e.g., physical disk 501A, in step 604. 

501A. If RAID controller 307 received a request to read data If the updating of the firmware is complete, then, in step 

stored in a stripe unit 502, e.g., stripe unit 502A, in the 55 611, a determination may be made as to whether any stripes 

selected disk 501, e.g., physical disk 501A, then, in step 605, 503 were updated. If there were no stripes 503 updated, then, 

RAID controller 307 may retrieve and transmit the requested in step 612, there is no need to rebuild any stripe units 502, 

data stored in stripe unit 502, e.g., stripe unit 502F, that e.g., stripe units 502A-E, in the selected disk 501, e.g., disk 

mirrors stripe unit 502, e.g., stripe unit 502A, containing the 501 A. 

requested data. For example, if client 110, e.g., client 110A, 60 If there were stripes 503 updated, then a stripe unit 502, 

requested to read the data labeled "A" in stripe unit 502A e.g., stripe unit 502A, associated with a stripe 503 updated 

during updating of the firmware on physical disk SOLA, then may be rebuilt in step 613. In one embodiment, stripe units 

RAID controller 307 may retrieve and transmit the data 502, e.g., stripe unit 502A, in the selected disk 501, e.g., disk 

labeled "A" in stripe unit 502F that mirrors stripe unit 502 A. 501 A, associated with stripes 503 that have been updated 

If RAID controller 307 did not receive a request to read 65 may be rebuilt stripe by stripe 503 updated, e.g., stripe 

data stored in stripe unit 502, e.g., stripe unit 502A, in the 503A, stripe 503D, starting from the top stripe 503 that was 

selected disk 501, e.g., physical disk 501A, then, in step 606, updated, e.g., stripe 503A, to the bottom stripe 503, e.g., 
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stripe 503D, if any, that was updated. In one embodiment, In step 619, a determination may be made as to whether 

the stripe unit 502, e.g., stripe unit 502A, associated with a the rebuilding of the stripe units 502, e.g., stripe units 502A, 

stripe 503 updated may be rebuilt by copying data in the stripe units 502D, whose data has changed has been com- 

stripe unit 502, e.g., stripe unit 502F, that mirrors the stripe pleted. If not, then the next stripe unit 502, e.g., stripe unit 

unit 502, e.g., stripe unit 502A, to be rebuilt and storing that 5 5Q2D, required to be rebuilt may be rebuilt in step 613. If the 

data in the stripe unit 502, e.g., stripe unit 502A, to be rebuilding of the stripe units 502, e.g., stripe units 502A, 

rebuilt stripe units 502D, in the selected disk 501, e.g., physical disk 

In step 614, each stripe 503, e.g., stripe 503A, associated 501A, whose data has changed has been completed, then 

with the stripe unit 502, e.g., stripe unit 502A, rebuilt may me T th( ? d 60 ° ^ terminated in step 620 

be tracked. In one embodiment, stripes 503 associated with 10 * * ° otcd ^ cv fl en xhaa & me ^ 60 ? /escribes a 

stripe units 502 rebuilt may be tracked in a table stored in ™ h i £ ^ atmB ^Taw , a d f k m 5 W 

r . .A. / nAm t „ - n _ system 130 implementing a RAID level one system without 

non-volatile memory 405 in RAID controller 307. deactivating RAID controller 307 that the principles 

In step 615 a determination may be made as to whether described in method 600 may be applicable to any redundant 
RAID controller 307 received a write request for a stripe unit level h fe DOted that a person of oldinary sJdl) 

502, e.g., stripe unit 502A, rebuilt in the selected disk 501, 15 would ^ capa ble of applying the principles taught in 
e.g., disk 501A, prior to completing the rebuilding of all method 60 o to any rcdundant RAID level. It is further noted 
stripe units 502 required to be rebuilt. that method 600 may be executed in a different order 

If RAID controller 307 did not receive a write request for presented and that the order presented in the discussion of 
a stripe unit 502, e.g., stripe unit 502A, rebuilt in the selected FIG. 6 is illustrative. It is further noted that certain steps in 
disk 501, e.g., disk 501A, prior to completing the rebuilding 20 method 600 may be executed almost concurrently, 
of all stripe units 502, e.g., stripe unit 502D, required to be FIG. 7 — RAID Level Five 

rebuilt, then, in step 617, a determination may be made as to FIG. 7 illustrates RAID storage system 130 implementing 
whether another disk 501 in the disk array of RAID storage RAID level five where RAID level five uses data interleav- 
system 130 has become inoperative during the process of ing by striping data across all disks and provides for error 
rebuilding the stripe units 502 in the selected disk 501 25 correction by distributing parity data across all disks. Refer- 
required to be rebuilt as discussed further below. ring to FIG. 7, the disk array in RAID storage system 130 

If RAID controller 307 received a write request for a implementing RAID level five may comprise a plurality of 
stripe unit 502, e.g., stripe unit 502A, rebuilt in the selected physical disks 701A-D configured to store data including 
disk 501, e.g., disk 501A, prior to completing the rebuilding parity data. Physical disks 701A-D may collectively or 
of all stripe units 502, e.g., stripe unit 502D, required to be 30 individually be referred to as physical disks 701 or physical 
rebuilt, then, in step 616, the updated data may be written in disk 701. It is noted that the disk array of RAID storage 
both the stripe unit 502, e.g., stripe unit 502A, rebuilt in the system 130 implementing RAID level five may comprise 
selected disk 501, e.g., disk 501 A, and in the stripe unit 502, any number of physical disks 701 and that FIG. 7 is 
e.g., stripe unit 502F, that mirrors the stripe unit 502, e.g., illustrative. 

stripe unit 502A, rebuilt. For example, if stripes 503A and 35 Each physical disk 701 may comprise a plurality of units 
503D were indicated as being rebuilt, then stripe units 502A commonly referred to as stripe units 702A-L. For example, 
and 502D need to be updated with the updated data. If after physical disk 701 A may comprise stripe units 702 A-C. 
stripe unit 502 A is updated and prior to stripe unit 502D Physical disk 701 B may comprise stripe units 702D-F and 
being updated, a request to write data to stripe unit 502 A is so forth. Stripe units 702 A-L may collectively or individu- 
received, then RAID controller 307 may write the updated 40 ally be referred to as stripe units 702 or stripe unit 702, 
data in stripe unit 502, e.g., stripe unit 502 A, already rebuilt respectively. Each stripe unit 702 may be configured to store 
in selected disk 501, e.g., disk 501A, as well as in the stripe a predefined amount, e.g., bit(s), byte(s), of data including 
unit 502, e.g., stripe unit 502F, that mirrors the stripe unit parity data. Stripe units 702 that are labeled with a "D" refers 
502, e.g., stripe unit 502 A, rebuilt whose data was overwrit- to stripe units 702 that store non-parity data. Stripe units 702 
ten with the updated data. 45 that are labeled with a "P" refers to stripe units 702 that store 

In step 617, a determination may be made as to whether parity data. It is noted that each physical disk 701 may 
another disk 501 in the disk array of RAID storage system comprise any number of stripe units 702. 
130 has become inoperative during the process of rebuilding A stripe 703 A-C may refer to a group of corresponding 
the stripe units 502 in selected disk 501 required to be stripe units 702. For example, stripe 703A may include 
rebuilt. 50 stripe units Dl, D2, D3 and PI. Stripe 703B may include 

If another disk 501, e.g., disk 501B, in the disk array of stripe units P2, D4, D5 and D6. Stripe 703C may include 
RAID storage system 130 does not become inoperative stripe units D7, P3, D8 and D9. Stripes 703 A-C may 
during the process of rebuilding the stripe units 502 in the collectively or individually be referred to as stripes 703 or 
selected disk 501 required to be rebuilt, then a determination stripe 703. 

may be made in step 619 as to whether the rebuilding of the 55 The disk array in RAID storage system 130 may include 
stripe units 502, e.g., stripe units 502 A, stripe units 502D, logical drives which divide physical drives 701 in the disk 
whose data has changed has been completed as discussed array into logical components which may be viewed by 
further below. client 110 as separate drives. Each logical drive may include 

If another disk 501, e.g., disk 501B, in the disk array of a cross section of each of physical drives 701. For example, 
RAID storage system 130 has become inoperative during the 60 logical drive 704 may include stripes 703 A-C across physi- 
process of rebuilding the stripe units 502 in the selected disk cal drives 701 A-D. 

501 required to be rebuilt, then once that disk 501, e.g., disk FIG. 8 — Method for Updating Firmware on a Disk In a 
501B, becomes operative, the stripe units 502, e.g., stripe RAID Level Five System 

units 502A, stripe units 502D, in the selected disk 501, e.g., FIG. 8 is a flowchart of one embodiment of the present 
disk 501A, required to be rebuilt may be rebuilt, in step 618, 65 invention of a method 800 for updating firmware on a disk 
using a copy of the updated data stored for backup purposes in RAID storage system 130 implementing a RAID level 
in step 608. five system without deactivating server 120. 
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Referring to FIGS. 7 and 8, in step 801, a particular 
physical disk 701, e.g., physical disk 701A, in RAID storage 
system 130 (FIGS. 1 and 3) implementing a RAID level five 
system may be selected to update the firmware in that 
particular disk 501. 5 

In step 802, the selected disk 701, e.g., physical disk 
70LA, as well as the associated logical drives 704 (FIG. 7) 
may enter a degrade mode of operation. In the degrade mode 
of operation, RAID controller 307 (FIGS. 3 and 4) may 
suppress particular activities, e.g., recovery actions, hot to 
spare kickin, from occurring on the selected disk 701 as well 
as prevent requests, e.g., read/write requests issued from 
client 110 (FIG. 1), hot swap queries, from reaching the 
selected disk 701. By suppressing particular activities from 
occurring on the selected disk 701 and preventing requests 15 
from reaching selected disk 701, activity on the selected disk 

701 may become inactive thereby allowing the firmware on 
the selected disk 701 to be updated. 

In step 803, the firmware on the selected disk 701, e.g., 
physical disk 701A, may be updated. 20 
During the updating of the firmware on the selected disk 

701, e.g., physical disk 701A, the following steps 804-812 
may occur. In step 804, a determination may be made as to 
whether RAID controller 307 received any read requests for 
stripe units 702 in the selected disk 701, e.g., physical disk 25 
701A. If RAID controller 307 received a request to read data 
stored in a stripe unit 702, e.g., stripe unit 702 A, in the 
selected disk 701, e.g., physical disk 701 A, then, in step 805, 
RAID controller 307 may perform a logical calculation on 
data, e.g., D2, D3 and PI, located in other stripe units 702, 30 
e.g., stripe units 702D, 702G and 702J, of stripe 703, e.g., 
stripe 703 A, associated with the stripe unit 702, e.g., stripe 
unit 702A, containing the data requested. In one 
embodiment, the logical calculation may be an exclusive or 
(XOR) calculation performed by DRAM/XOR controller 35 
408 (FIG. 4). For example, the data requested in stripe unit 

702, e.g., stripe unit 702 A, in the selected disk 701, e.g., 
physical disk 701A, may be calculated by performing the 
XOR function on D2, D3 and PI. The resulting data may 
then be transmitted to the requesting client 110 in step 806. 40 

If RAID controller 307 did not receive a request to read 
data stored in stripe unit 702, e.g., stripe unit 702A, in the 
selected disk 701, e.g., physical disk 701A, then, in step 807, 
a determination may be made as to whether RAID controller 
307 received any write requests for stripe units 702 in the 45 
selected disk 701, e.g., physical disk 701 A. Furthermore, 
upon transmitting the requested data in step 806, a determi- 
nation may be made in step 807 as to whether RAID 
controller 307 received any write requests for stripe units 

702 in the selected disk 701, e.g., physical disk 701 A. 50 
If RAID controller 307 did not receive any write requests 

for stripe units 702 in the selected disk 701, e.g., physical 
disk 701A, then a determination may be made in step 812 as 
to whether the updating of the firmware is complete as 
discussed further below. 55 

If RAID controller 307 received any write requests for 
stripe units 702 in the selected disk 701, e.g., physical disk 
701A, then, in step 808, RAID controller 307 may generate 
a new parity for the stripe 703, e.g., stripe 703 A, associated 
with stripe unit 702, e.g., stripe unit 702 A, containing 60 
outdated data. A new parity for the stripe 703, e.g., stripe 
703A, associated with the stripe unit 702, e.g., stripe unit 
702A, containing outdated data may be generated so that the 
updated data may replace the outdated data as explained 
further below. For example, if client 110 requests to write 65 
data Dl in stripe unit 702A during the updating of the 
firmware on disk 501A, then a new parity PI may be 



generated for stripe 703A associated with stripe unit 702 A. 
In one embodiment, a new parity may be generated by 
performing a logical operation on the data to be written 
along with the data stored in other stripe units 702, e.g., 
stripe unit 702B, stripe unit 702C, except the stripe unit 702, 
e.g., stripe unit 702J, storing the parity data to be updated 
and the stripe unit 702, e.g., stripe unit 702 A, of the selected 
disk 701, e.g., disk 701 A, whose firmware is being updated. 
In one embodiment, the logical calculation may be an 
exclusive or (XOR) calculation performed by DRAM/XOR 
controller 408. For example, if the data to be written over Dl 
was X, then a new parity may be generated for PI by 
performing the XOR function on X, D2 and D3. The updated 
parity may then replace the older parity associated the stripe 
703, e.g., 703A, updated in step 809. 

In step 810, a copy of the updated data, i.e., the data 
requested to be written in stripe units 702 in the selected disk 
701, may be stored for backup purposes. In one 
embodiment, a copy of the updated data may be stored in 
either a non-volatile cache 407 (FIG. 4) in RAID controller 
307 or in a special storage area typically called a log area in 
a limited number, e.g., two, of other drives 701, e.g., disk 
701B, disk 701C. It is noted for clarity that each drive 701 
may be capable of containing a special storage area typically 
called a log area. 

In step 811, stripe 703, e.g., stripe 703A, associated with 
stripe unit 702, e.g., stripe unit 702A, in the selected disk 

701, e.g., physical disk 701 A, whose data was changed may 
be tracked. In one embodiment, stripes 703 associated with 
stripe units 702 in selected disk 701 whose data was changed 
may be tracked in a table stored in non-volatile memory 405 
(FIG. 4) of RAID controller 307. 

In step 812, a determination may be made if the updating 
of the firmware on the selected disk 701, e.g., physical disk 
701A, is completed. Upon completion of the firmware being 
updated on the selected disk 701, e.g., physical disk 701 A, 
the stripe units 702 in the selected disk 701, e.g., physical 
disk 701A, associated with any stripes to be updated, may be 
rebuilt as described further below in steps 813-318. 

If the updating of the firmware is not completed, then a 
determination may be made as to whether RAID controller 
307 received any read requests for stripe units 702 in the 
selected disk 701, e.g., physical disk 701A, in step 804. 

If the updating of the firmware is complete, then, in step 
813, a determination may be made as to whether any stripes 
703 were updated. If there were no stripes 703 updated, then, 
in step 814, there is no need to rebuild any stripe units 702, 
e.g., stripe units 702A-C, in the selected disk 701, e.g., disk 
701A. 

If there were stripes 703 updated, then a stripe unit 702, 
e.g., stripe unit 702A, associated with a stripe 703 updated 
may be rebuilt in step 815. In one embodiment, stripe units 

702, e.g., stripe unit 702A, in the selected disk 701, e.g., disk 
701A, associated with stripes 703 that have been updated 
may be rebuilt stripe by stripe 703 updated, e.g., stripe 
703 A, starting from the top stripe 703 that was updated, e.g., 
stripe 703 A, to the bottom stripe 703, if any, that was 
updated. In one embodiment, the stripe unit 702, e.g., stripe 
unit 702A, associated with a stripe 703 updated may be 
rebuilt by performing a logical calculation on data, e.g., D2, 
D3 and PI, located in other stripe units 702, e.g., stripe units 
702D, 702G and 702J, of stripe 703, e.g., stripe 703A, 
associated with the stripe unit 702, e.g., stripe unit 702 A, 
containing the outdated data. In one embodiment, the logical 
calculation may be an exclusive or (XOR) calculation per- 
formed by DRAM/XOR controller 408. For example, the 
updated data to be inserted in the stripe unit 702, e.g., stripe 
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unit 702 A, in the selected disk 701, e.g., physical disk 701 A, 
to be rebuilt may be calculated by performing the XOR 
function on D2, D3 and PI. The resulting data may then be 
inserted in the stripe unit 702, e.g., stripe unit 703 A, to be 
rebuilt. 

In step 816, a determination may be made as to whether 
another disk 501 in the disk array of RAID storage system 
130 has become inoperative during the process of rebuilding 
the stripe units 702 in the selected disk 701 required to be 
rebuilt. 

If another disk 701, e.g., disk 701B, in the disk array of 
RAID storage system 130 does not become inoperative 
during the process of rebuilding the stripe units 702 in the 
selected disk 701 required to be rebuilt, then a determination 
may be made in step 818 as to whether the rebuilding of the 
stripe units 702, e.g., stripe units 502A, whose data has 
changed has been completed as discussed further below. 

If another disk 701, e.g., disk 701B, in the disk array of 
RAID storage system 130 has become inoperative during the 
process of rebuilding the stripe units 702 in the selected disk 20 
701 required to be rebuilt, then once that disk 701, e.g., disk 
701B, becomes operative, the stripe units 702, e.g., stripe 
units 702A, in the selected disk 701, e.g., disk 701 A, 
required to be rebuilt may be rebuilt, in step 817, using a 
copy of the updated data stored for backup purposes in step 25 
810. 

In step 818, a determination may be made as to whether 
the rebuilding of the stripe units 702, e.g., stripe units 702 A, 
whose data has changed has been completed. If not, then the 
next stripe unit 702, e.g., stripe unit 702B, required to be 
rebuilt may be rebuilt in step 815. If the rebuilding of the 
stripe units 702, e.g., stripe units 702A, in the selected disk 
701, e.g., physical disk 701 A, whose data has changed has 
been completed, then method 800 may be terminated in step 
819. 

It is noted that even though method 800 describes a 
method for updating firmware on a disk in RAID storage 
system 130 implementing a RAID level five system without 
deactivating server 120 that the principles described in 



method 800 may be applicable to any redundant RAID level. 40 ste P s 



updating said firmware in said selected disk; and 
tracking one or more stripes in said disk array correspond- 
ing to one or more of said plurality of stripe units of 
said selected disk where data in said one or more of said 
plurality of stripe units of said selected disk change in 
response to write requests to said selected disk. 

2. The method as recited in claim 1 further comprising the 
steps of: 

receiving a request to read from a stripe unit of said 

selected disk; and 
transmitting data from a stripe unit that mirrors said stripe 
unit of said selected disk, wherein said mirror stripe 
unit is located in another disk of said disk array. 

3. The method as recited in claim 1 further comprising the 
15 steps of: 

receiving a request to write data to a stripe unit of said 

selected disk; and 
writing said data to a stripe unit that mirrors said stripe 
unit of said selected disk, wherein said mirror stripe 
unit is located in another disk of said disk array. 

4. The method as recited in claim 3 further comprising the 
step of: 

storing a copy of said data for backup purposes. 

5. The method as recited in claim 1 further comprising the 
steps of: 

completing updating of said firmware in said selected 
disk; and 

rebuilding a stripe unit of said selected disk corresponding 
to a stripe tracked. 

6. The method as recited in claim 5, wherein said step of 
rebuilding said stripe unit of said selected disk comprises the 
steps of: 

copying data in a stripe unit in another disk of said disk 
array that mirrors said stripe unit of said selected disk 
to be rebuilt; and 

storing said copied data in said stripe unit of said selected 
disk to be rebuilt. 

7. The method as recited in claim 1 further comprising the 
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It is further noted that a person of ordinary skill would be 
capable of applying the principles taught in method 800 to 
any redundant RAID level. It is further noted that method 
800 may be executed in a different order presented and that 
the order presented in the discussion of FIG. 8 is illustrative. 
It is further noted that certain steps in method 800 may be 
executed almost concurrently. 

Although the system, method and computer program 
product are described in connection with several 
embodiments, it is not intended to be limited to the specific 
forms set forth herein, but on the contrary, it is intended to 
cover such alternatives, modifications and equivalents, as 
can be reasonably included within the spirit and scope of the 
invention as defined by the appended claims. It is noted that 
the headings are used only for organizational purposes and 55 
not meant to limit the scope of the description or claims. 

What is claimed is: 

1. A method for updating firmware in a disk in a disk array 
without deactivating a server coupled to said disk array 
comprising the steps of: 

selecting said disk in said disk array, wherein said selected 
disk comprises said firmware to be updated, wherein 
each disk in said disk array comprises a plurality of 
stripe units; 



60 



completing updating of said firmware in said selected 
disk; and 

rebuilding one or more stripe units of said selected disk 
corresponding to one or more stripes tracked stripe by 
stripe. 

8. The method as recited in claim 7 further comprising the 
step of: 

tracking which of said one or more stripes corresponding 
to which of said one or more stripe units have been 
rebuilt. 

9. The method as recited in claim 8 further comprising the 
step of: 

receiving a request to write data to a stripe unit of said 
selected disk that has been rebuilt prior to completing 
rebuilding of said one or more stripe units of said 
selected disk. 

10. The method as recited in claim 9 further comprising 
the step of: 

writing said data in said stripe unit of said selected disk 
that has been rebuilt and in a stripe unit that is a mirror 
of said stripe unit of said selected disk that has been 
rebuilt, wherein said mirror stripe unit is located in 
another disk of said disk array. 

11. The method as recited in claim 1 further comprising 



entering a degrade mode of operation by said selected 65 me ste P s °£ 
disk, wherein said degrade mode of operation prevents receiving a request to read from a stripe unit of a par- 
said selected disk from receiving requests; ticular stripe of said selected disk; and 
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performing a logical calculation on data located in other 
stripe units of said particular stripe. 

12. The method as recited in claim 11 further comprising 
the step of: 

transmitting a result of said logical calculation. 

13. The method as recited in claim 11, wherein said 
logical calculation is an exclusive OR calculation. 

14. The method as recited in claim 1 further comprising 
the steps of: 

receiving a request to write data to a stripe unit of a 
particular stripe of said selected disk; and 

generating an updated parity value stored in another stripe 
unit of said particular stripe. 

15. The method as recited in claim 14, wherein said 
updated parity value is generated by performing a logical 
calculation on said data to be written with data located in 
other stripe units of said particular stripe except data located 
in a stripe unit storing a parity value to be updated. 

16. The method as recited in claim 15, wherein said 
logical calculation is an exclusive OR calculation. 

17. The method as recited in claim 14 further comprising 
the step of: 

storing a copy of said data requested to be written for 
backup purposes. 

18. The method as recited in claim 14 further comprising 
the step of: 

tracking said particular stripe that was updated. 

19. The method as recited in claim 18 further comprising 
the steps of: 

completing updating of said firmware in said selected 
disk; and 

rebuilding said stripe unit of said selected disk corre- 
sponding to said particular stripe that was updated. 

20. The method as recited in claim 19, wherein said step 
of rebuilding is accomplished by performing a logical cal- 
culation on data located in other stripe units of said particu- 
lar stripe. 

21. A system, comprising: 

an array of disks, wherein each disk in a disk array 

comprises a plurality of stripe units; and 
a controller coupled to said array of disks, wherein said 

controller is configured to manage said array of disks, 

wherein said controller comprises: 

a memory unit operable for storing a program for 
updating firmware in a disk in said disk array without 
deactivating a server coupled to said array of disks, 
wherein said server is configured to receive requests 
to access said array of disks, wherein said server is 
configured to forward said received requests to said 
controller; 

a processor coupled to said memory, wherein said 
processor, responsive to said program, comprises: 
circuitry operable for selecting a disk in said disk 
array, wherein said selected disk comprises said 
firmware to be updated; 
circuitry operable for entering a degrade mode of 
operation for said selected disk, wherein said 
degrade mode of operation prevents said selected 
disk from receiving requests; 
circuitry operable for updating said firmware in said 

selected disk; and 
circuitry operable for tracking one or more stripes in 
said disk array corresponding to one or more of 
said plurality of stripe units of said selected disk 
where data in said one or more of said plurality of 
stripe units of said selected disk change in 
response to write requests to said selected disk. 
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22. The system as recited in claim 21, wherein said 
controller further comprises: 

circuitry operable for receiving a request to read from a 
stripe unit of said selected disk; and 
5 circuitry operable for transmitting data from a stripe unit 
that mirrors said stripe unit of said selected disk, 
wherein said mirror stripe unit is located in another disk 
of said disk array. 

23. The system as recited in claim 21, wherein said 
1Q controller further comprises: 

circuitry operable for receiving a request to write data to 

a stripe unit of said selected disk; and 
circuitry operable for writing said data to a stripe unit that 

mirrors said stripe unit of said selected disk, wherein 
15 said mirror stripe unit is located in another disk of said 

disk array. 

24. The system as recited in claim 23, wherein said 
controller further comprises: 

a non-volatile cache coupled to said processor, wherein 
2Q said non-volatile cache is configured to store a copy of 
said data for backup purposes. 

25. The system as recited in claim 21, wherein said 
controller further comprises: 

circuitry operable for completing updating of said firm- 
25 ware in said selected disk; and 

circuitry operable for rebuilding a stripe unit of said 
selected disk corresponding to a stripe tracked, 

26. The system as recited in claim 25, wherein said 
circuitry operable for rebuilding said stripe unit of said 

3Q selected disk comprises: 

circuitry operable for copying data in a stripe unit in 
another disk of said disk array that mirrors said stripe 
unit of said selected disk to be rebuilt; and 
circuitry operable for storing said copied data in said 
stripe unit of said selected disk to be rebuilt, 
35 27. The system as recited in claim 21, wherein said 
controller further comprises: 

circuitry operable for completing updating of said firm- 
ware in said selected disk; and 
circuitry operable for rebuilding one or more stripe units 
of said selected disk corresponding to one or more 
stripes tracked stripe by stripe tracked. 

28. The system as recited in claim 27, wherein said 
controller further comprises: 

circuitry operable for tracking which of said one or more 
stripes corresponding to which of said one or more 
stripe units have been rebuilt. 

29. The system as recited in claim 28, wherein said 
controller further comprises: 

circuitry operable for receiving a request to write data to 
50 a stripe unit of said selected disk that has been rebuilt 
prior to completing rebuilding of said one or more 
stripe units of said selected disk. 

30. The system as recited in claim 29, wherein said 
controller further comprises: 

55 circuitry operable for writing said data in said stripe unit 
of said selected disk that has been rebuilt and in a stripe 
unit that is a mirror of said stripe unit of said selected 
disk that has been rebuilt, wherein said mirror stripe 
unit is located in another disk of said disk array. 

60 31. The system as recited in claim 21, wherein said 
controller further comprises: 

circuitry operable for receiving a request to read from a 
stripe unit of a particular stripe of said selected disk; 
and 

65 circuitry operable for performing a logical calculation on 
data located in other stripe units of said particular 
stripe. 



06/17/2004, EAST Version: 1.4.1 



US 6,728 : 

21 

32. The system as recited in claim 31, wherein said 
controller further comprises: 

circuitry operable for transmitting a result of said logical 
calculation. 

33 . The system as recited in claim 31, wherein said logical 5 
calculation is an exclusive OR calculation. 

34. The system as recited in claim 21, wherein said 
controller further comprises: 

circuitry operable for receiving a request to write data to 
a stripe unit of a particular stripe of said selected disk; ™ 
and 

circuitry operable for generating an updated parity value 
stored in another stripe unit of said particular stripe. 

35. The system as recited in claim 34, wherein said 
updated parity value is generated by performing a logical 15 
calculation on said data to be written with data located in 
other stripe units of said particular stripe except data located 

in a stripe unit storing a parity value to be updated. 

36. The system as recited in claim 35, wherein said logical 
calculation is an exclusive OR calculation. 20 

37. The system as recited in claim 34, wherein said 
controller further comprises: 

a non-volatile cache coupled to said processor, wherein 
said non-volatile cache is configured to store a copy of 
said data requested to be written for backup purposes. 25 

38. The system as recited in claim 34, wherein said 
controller stores a copy of said data requested to be written 
in a special area in a limited number of other disks of said 
disk array. 

39. The system as recited in claim 34, wherein said 30 
controller further comprises: 

circuitry operable for tracking said particular stripe that 
was updated. 

40. The system as recited in claim 39, wherein said 
controller further comprises: 35 

circuitry operable for completing updating of said firm- 
ware in said selected disk; and 

circuitry operable for rebuilding said stripe unit of said 
selected disk corresponding to said particular stripe that 
was updated. 40 

41. The system as recited in claim 40, wherein said 
circuitry operable for rebuilding is accomplished by per- 
forming a logical calculation on data located in other stripe 
units of said particular stripe. 

42. A system, comprising: 45 
means for selecting said disk in said disk array, wherein 

said selected disk comprises said firmware to be 
updated, wherein each disk in said disk array comprises 
a plurality of stripe units; 

means for entering a degrade mode of operation by said 50 
selected disk, wherein said degrade mode of operation 
prevents said selected disk from receiving requests; 

means for updating said firmware in said selected disk; 
and 

means for tracking one or more stripes in said disk array 55 
corresponding to one or more of said plurality of stripe 
units of said selected disk where data in said one or 
more of said plurality of stripe units of said selected 
disk change in response to write requests to said 
selected disk. 60 

43. The system as recited in claim 42 further comprises: 
means for receiving a request to read from a stripe unit of 

said selected disk; and 
means for transmitting data from a stripe unit that mirrors 
said stripe unit of said selected disk, wherein said 65 
mirror stripe unit is located in another disk of said disk 
array. 
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44. The system as recited in claim 42 further comprises: 
means for receiving a request to write data to a stripe unit 

of said selected disk; and 
means for writing said data to a stripe unit that mirrors 
said stripe unit of said selected disk, wherein said 
mirror stripe unit is located in another disk of said disk 
array. 

45. The system as recited in claim 44 further comprises: 
means for storing a copy of said data for backup purposes. 

46. The system as recited in claim 42 further comprises: 
means for completing updating of said firmware in said 

selected disk; and 
means for rebuilding a stripe unit of said selected disk 
corresponding to a stripe tracked. 

47. The system as recited in claim 46, wherein said means 
for rebuilding said stripe unit of said selected disk com- 
prises: 

means for copying data in a stripe unit in another disk of 
said disk array that mirrors said stripe unit of said 
selected disk to be rebuilt; and 

means for storing said copied data in said stripe unit of 
said selected disk to be rebuilt. 

48. The system as recited in claim 42 further comprises: 
means for completing updating of said firmware in said 

selected disk; and 
means for rebuilding one or more stripe units of said 
selected disk corresponding to one or more stripes 
tracked stripe by stripe. 

49. The system as recited in claim 48 further comprises: 
means for tracking which of said one or more stripes 

corresponding to which of said one or more stripe units 
have been rebuilt. 

50. The system as recited in claim 49 further comprises: 
means for receiving a request to write data to a stripe unit 

of said selected disk that has been rebuilt prior to 
completing rebuilding of said one or more stripe units 
of said selected disk. 

51. The system as recited in claim 50 further comprises: 
means for writing said data in said stripe unit of said 

selected disk that has been rebuilt and in a stripe unit 
that is a mirror of said stripe unit of said selected disk 
that has been rebuilt, wherein said mirror stripe unit is 
located in another disk of said disk array. 

52. The system as recited in claim 42 further comprises: 
means for receiving a request to read from a stripe unit of 

a particular stripe of said selected disk; and 
means for performing a logical calculation on data located 
in other stripe units of said particular stripe. 

53. The system as recited in claim 52 further comprises: 
means for transmitting a result of said logical calculation. 

54. The system as recited in claim 52, wherein said logical 
calculation is an exclusive OR calculation. 

55. The system as recited in claim 42 further comprises: 
means for receiving a request to write data to a stripe unit 

of a particular stripe of said selected disk; and 
means for generating an updated parity value stored in 
another stripe unit of said particular stripe, 

56. The system as recited in claim 55, wherein said 
updated parity value is generated by performing a logical 
calculation on said data to be written with data located in 
other stripe units of said particular stripe except data located 
in a stripe unit storing a parity value to be updated. 

57. The system as recited in claim 56, wherein said logical 
calculation is an exclusive OR calculation. 

58. The system as recited in claim 55 further comprises: 
means for storing a copy of said data requested to be 

written for backup purposes. 



06/17/2004, EAST Version: 1.4.1 



US 6,728,833 B2 

23 24 

59. The system as recited in claim 55 further comprises: 69. The computer program product as recited in claim 68 
means for tracking said particular stripe that was updated. further comprising the programming step of: 

60. The system as recited in claim 59 further comprises: tracking which of said one or more stripes corresponding 
means for completing updating of said firmware in said to which of said one or more stripe units have been 

selected disk; and 5 rebuilt, 

means for rebuilding said stripe unit of said selected disk 70. The computer program product as recited in claim 69 

corresponding to said particular stripe that was further comprising the programming step of: 

u P datcd- . . . receiving a request to write data to a stripe unit of said 

a. The system as reeled m claim 60, wherem said means tha , ^ ^ rebuU , ^ ^p^g 

for rebuilduig * accomplished by performing a logical 10 ^ ^ ^ ^ ^ ^ ^ ^ *- J 

calculation on data located in other stripe units of said selected disk 

particular stripe. ^ com p Uter program product as recited in claim 70 

62. A computer program product embodied in a machine . .if • . r 
j i * f. r r j *• a - j * i * j- i further comprising the programming step of: 

readable medium for updating firmware in a disk in a disk ... 

array without deactivating a server coupled to said disk array 15 writing said data in said stripe unit of said selected disk 

comprising the programming steps of: mat has been rebuilt and ^ a stri P e unit mat * a mirror 

selecting said disk in said disk array, wherein said selected unit ° f selected disk lhat , has ^ 

disk comprises said firmware to be updated, wherein rebuilt ' wherem said mirror stn P e ^ 15 locatcd 10 

each disk in said disk array comprises a plurality of another dlsk of ™ ld disk arrav - 

stripe units; 20 72. The computer program product as recited in claim 62 

entering a degrade mode of operation by said selected further comprising the programming steps of: 
disk, wherein said degrade mode of operation prevents receiving a request to read from a stripe unit of a par- 
said selected disk from receiving requests; ticular stripe of said selected disk; and 

updating said firmware in said selected disk; and performing a logical calculation on data located in other 

tracking one or more stripes in said disk array correspond- 25 stripe units of said particular stripe, 

ing to one or more of said plurality of stripe units of 73. The computer program product as recited in claim 72 

said selected disk where data in said one or more of said further comprising the programming step of: 

plurality of stripe units of said selected disk change in „ . . lt - ... . . . . . 

respond to write requests to said selected disk. transmitting a result of said logical calcu anon. 

63. The computer program product as recited in claim 62 30 74. Tlie computer program product as recited m claim 72, 
further comprising the programming steps of: wherein said logical calculation is an exclusive OR calcu- 



receiving a request to read from a stripe unit of said 



lation. 



selected disk* and 75. The computer program product as recited in claim 72 

transmitting data from a stripe unit that mirrors said stripe furlher comprising the programming steps of: 

unit of said selected disk, wherein said mirror stripe 35 receiving a request to write data to a stripe unit of a 

unit is located in another disk of said disk array. particular stripe of said selected disk; and 

64. The computer program product as recited in claim 62 generating an updated parity value stored in another stripe 
further comprising the programming steps of: unit of said particular stripet 

receiving a request to write data to a stripe unit of said 75. The computer program product as recited in claim 75, 

selected disk; and 40 wherem upc ]ated parity value is generated by perform- 

writing said data to a stripe unit that mirrors said stripe ing a logical ca i cu i ation on said data to be written with data 

unit of said selected disk, wherein said mirror stripe , ocated m othcr strf units of &aid iculaf ^ 

unit is located in another disk of said disk array. , t . 4 , . f . . . r . f * 

65. The computer program product as recited in claim 64 ^ a ! °? ated lD a strl P 6 umt stolm & » P*my value to be 

further comprising the programming step of: 45 U ^?i e rT ^ , m , 

. _ . j . ° f . , 77. The computer program product as recited in claim 76, 

storing a copy of said data for backup purposes. wherein said lotrical calculation is an exclusive OR calcu 

66. The computer program product as recited in claim 62 ^o n calculation is an exclusive OR calcu- 

further comprising the programming steps of: _ 0 *„, . , A , . , . „ 

i I • i ? .j n • j i j 78. The computer program product as recited in claim 75 

"dS Tnd WarC 10 ^ 50 further comprising the programming step of: 

rebuadingastripeunitofsaidselecteddiskcorresponding sto u rin i a ^ of said data guested to be written for 

to a stripe tracked. purposes. 

67. The computer program product as recited in claim 66, A 7 U 9 ' ™ e computer program product as recited in claim 75 
wherein said programming step of rebuilding said stripe unit furlher comprising the programming step of: 

of said selected disk comprises the programming steps of: 55 tracking said particular stripe that was updated. 

copying data in a stripe unit in another disk of said disk 80 ' ^ computer program product as recited in claim 79 

array that mirrors said stripe unit of said selected disk furthcr comprising the programming steps of: 

to be rebuilt; and completing updating of said firmware in said selected 

storing said copied data in said stripe unit of said selected d isk; and 

disk to be rebuilt. 60 rebuilding said stripe unit of said selected disk corre- 

68. The computer program product as recited in claim 62 sponding to said particular stripe that was updated, 
further comprising the programming steps of: 81. The computer program product as recited in claim 80, 

completing updating of said firmware in said selected wherein said programming step of rebuilding is accom- 

disk; and plished by performing a logical calculation on data located 

rebuilding one or more stripe units of said selected disk 65 in other stripe units of said particular stripe, 
corresponding to one or more stripes tracked stripe by 

stripe. ***** 
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